Papers by Marta R. Costa-jussà
Measuring the Mixing of Contextual Information in the Transformer (2022.emnlp-main)
Copied to clipboard
| Challenge: | Experimentally, we show that ALTI provides more faithful explanations and increased robustness than gradient-based methods. |
| Approach: | They propose to measure token-to-token interactions within each layer and then use them to aggregate model predictions. |
| Outcome: | The proposed method provides more faithful explanations and increased robustness than gradient-based methods. |
Multilingual Machine Translation: Closing the Gap between Shared and Language-specific Encoder-Decoders (2021.eacl-main)
Copied to clipboard
| Challenge: | State-of-the-art multilingual machine translation relies on a universal encoder-decoder, which requires retraining the entire system to add new languages. |
| Approach: | They propose an encoder-decoder approach that can be extended to new languages by learning their corresponding modules. |
| Outcome: | The proposed approach outperforms the universal encoder-decoder by 3.28 BLEU points on average while allowing to add new languages without retraining the rest of the modules. |
Attention Weights in Transformer NMT Fail Aligning Words Between Sequences but Largely Explain Model Predictions (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Using attention weights, we show that NMT models make alignment errors by relying on uninformative tokens from the source sequence. |
| Approach: | They propose to use attention weights to regulate alignment errors in NMT models . they propose methods that largely reduce the word alignment error rate compared to standard induced alignments from attention weighted tokens. |
| Outcome: | The proposed methods reduce the word alignment error rate compared to standard induced alignments from attention weights. |
Abusive language in Spanish children and young teenager’s conversations: data preparation and short text classification with contextual word embeddings (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing studies on how to automatically detect abusive short texts are gaining interest in the natural language processing community. |
| Approach: | They propose to use a contextual word embedding model to automatically detect abusive short texts for Spanish language. |
| Outcome: | The proposed model outperforms classical methods in the detection of abusive short texts for the spanish language. |
Automatic Spanish Translation of SQuAD Dataset for Multi-lingual Question Answering (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods to train multilingual QA systems are limited for other languages . cross-lingual learning is a technique that transfers knowledge from source to target language with fewer training data. |
| Approach: | They propose a translation method to translate the Stanford Question Answering Dataset to Spanish and a multilingual-BERT model to train Spanish QA systems. |
| Outcome: | The proposed method outperforms the previous benchmarks for cross-lingual extractive QA. |
Continual Lifelong Learning in Natural Language Processing: A Survey (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to continual learning (CL) are costly and time-consuming. |
| Approach: | They propose to examine the problem of continual learning in NLP through the lens of various NLP tasks and provide a critical review of existing methods. |
| Outcome: | The proposed methods are critical to the development of CL models and provide a critical review of existing methods and datasets. |
Improving Language and Modality Transfer in Translation by Character-level Modeling (2025.acl-long)
Copied to clipboard
| Challenge: | Current translation systems cover only 5% of the world's languages . expanding to the long-tail of low-resource languages requires data-efficient methods that rely on cross-lingual and cross-modal knowledge transfer. |
| Approach: | They propose a character-based approach to improve adaptability to new languages and modalities by using a teacher-student approach and parallel translation data to obtain a SONAR character-level encoder. |
| Outcome: | The proposed model outperforms subword-based models in speech-to-text translation on the FLEURS benchmark on 33 languages and achieves state-of-the-art generalizability to unseen languages. |
SpeechAlign: A Framework for Speech Translation Alignment Evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Speech-to-Speech and Speech- to-Text translation are currently dynamic areas of research. |
| Approach: | They propose a framework to evaluate source-target alignment in speech models . they introduce a speech gold alignment dataset and introduce two new metrics . |
| Outcome: | The proposed framework evaluates source-target alignment quality within speech models. |
From Bilingual to Multilingual Neural Machine Translation by Incremental Training (P19-2)
Copied to clipboard
| Challenge: | Existing approaches to multilingual neural machine translation are based on task specific models and the addition of one more language is only possible by retraining the whole system. |
| Approach: | They propose a training schedule that scales to more languages without modification of previous components. |
| Outcome: | The proposed training schedule shows close results to state-of-the-art in the WMT task. |
GeBioToolkit: Automatic Extraction of Gender-Balanced Multilingual Corpus of Wikipedia Biographies (2020.lrec-1)
Copied to clipboard
| Challenge: | a tool for extracting multilingual parallel corpora at sentence level with document and gender information from Wikipedia biographies. |
| Approach: | They propose a tool for extracting multilingual parallel corpora at sentence level with document and gender information from Wikipedia biographies. |
| Outcome: | The proposed tool extracts a corpus of 2,000 sentences in English, Spanish and Catalan. |
Towards Opening the Black Box of Neural Machine Translation: Source and Target Interpretations of the Transformer (2022.emnlp-main)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) relies on source sentence and target prefix attributions for each input token. |
| Approach: | They propose an interpretability method that tracks input tokens’ attributions for both contexts and extends it to any encoder-decoder Transformer-based model. |
| Outcome: | The proposed method can be extended to any encoder-decoder Transformer-based model and provides insights into their behaviour. |
BOUQuET : dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translation (2025.emnlp-main)
Copied to clipboard
Pierre Andrews, Mikel Artetxe, Mariano Coria Meglioli, Marta R. Costa-jussà, Joe Chuang, David Dale, Mark Duppenthaler, Nathanial Paul Ekberg, Cynthia Gao, Daniel Edward Licht, Jean Maillard, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Eduardo Sánchez, Ioannis Tsiamas, Arina Turkatenko, Albert Ventayol-Boada, Shireen Yates
| Challenge: | BOUQUET is a multi-way, multicentric and multi-register/domain dataset and benchmark . the dataset is handcrafted in 8 non-English languages . |
| Approach: | They propose to use BOUQuET to collect a multi-way, multicentric and multi-register/domain dataset and benchmark in 8 non-English languages. |
| Outcome: | The proposed dataset is available at https://huggingface.co/datasets/facebook/bouquet. |
Evaluating Gender Bias in Speech Translation (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing evaluation techniques for gender biases are lacking in the field of machine translation. |
| Approach: | They propose to use a free evaluation set to evaluate gender bias in speech translation. |
| Outcome: | The proposed set is the speech version of WinoMT, an MT challenge set. |
Detecting and Mitigating Hallucinations in Machine Translation: Model Internal Workings Alone Do Well, Sentence Similarity Even Better (2023.acl-long)
Copied to clipboard
| Challenge: | a recent study shows that without artificially encouraging models to hallucinate, existing methods fall short . hallucinations are cases when the model generates output that is partially or fully unrelated to the source sentence. |
| Approach: | They propose a method that evaluates the percentage of the source contribution to a generated translation. |
| Outcome: | The proposed method improves detection accuracy for the most severe hallucinations by a factor of 2. |
On the Locality of Attention in Direct Speech Translation (2022.acl-srw)
Copied to clipboard
| Challenge: | Recent advances in NLP have created problems with the complexity of the self-attention layer. |
| Approach: | They propose to substitute standard self-attention with a local efficient one to avoid the computation of attention weights. |
| Outcome: | The proposed model matches the baseline performance and improves efficiency by skipping the computation of weights that standard attention discards. |
Multilingual, Multi-scale and Multi-layer Visualization of Intermediate Representations (D19-3)
Copied to clipboard
| Challenge: | Currently, the main alternatives to deal with sequences are Recurrent Neural Networks (RNN) architectures and the Transformer. |
| Approach: | They propose a web-based tool that visualizes the sentence and token representations of RNNs and Transformer architectures at the sentence level. |
| Outcome: | The proposed visualization tool analyses gender inequalities in contextual word embeddings and the common language representation in a multilingual machine translation system. |
LCFO: Long Context and Long Form Output Dataset and Benchmarking (2025.findings-acl)
Copied to clipboard
Marta R. Costa-jussà, Pierre Andrews, Mariano Coria Meglioli, Joy Chen, Joe Chuang, David Dale, Christophe Ropers, Alexandre Mourachko, Eduardo Sánchez, Holger Schwenk, Tuan A. Tran, Arina Turkatenko, Carleigh Wood
| Challenge: | Using long text outputs to evaluate progress in summarization and summary expansion tasks is challenging. |
| Approach: | They propose a framework for assessing gradual summarization and summary expansion capabilities across diverse domains. |
| Outcome: | The proposed framework provides alignments between specific QA pairs and corresponding summaries in 7 domains. |
Explaining How Transformers Use Context to Build Predictions (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for analyzing input attributions for a model's prediction are unclear how prior words affect the model' s decision throughout the layers. |
| Approach: | They propose a procedure to analyze models for language generation using the Transformer and a comparison of their results with evidence of the linguistic phenomena. |
| Outcome: | The proposed method consistently aligns better than gradient-based and perturbation-based baselines and generates human-like source-target alignments for building predictions. |
Combining Subword Representations into Word-level Representations in the Transformer Architecture (2020.acl-srw)
Copied to clipboard
| Challenge: | Currently dominant approaches use word-level tokens, but this increases the length of the sequences and makes it difficult to profit from word-based information. |
| Approach: | They propose to combine subword-level representations into word-level ones in the first layers of the encoder, reducing the effective length of the sequences in the following layers. |
| Outcome: | The proposed model maintains translation quality with no extra word-level information . it is superior to the current dominant method for incorporating word- level source language information a priori . |
BLASER: A Text-Free Speech-to-Speech Translation Evaluation Metric (2023.acl-long)
Copied to clipboard
Mingda Chen, Paul-Ambroise Duquenne, Pierre Andrews, Justine Kao, Alexandre Mourachko, Holger Schwenk, Marta R. Costa-jussà
| Challenge: | End-to-End speech-to speech translation is generally evaluated with text-based metrics . this means generated speech has to be automatically transcribed, making the evaluation dependent on ASR systems. |
| Approach: | They propose a text-free evaluation metric for end-to-end speech-tospeech translation, named BLASER, to avoid the dependency on automatic speech recognition systems. |
| Outcome: | The proposed metric avoids the dependency on automatic speech recognition systems by encoding generated speech segments into a shared embedding space. |
On the Role of Speech Data in Reducing Toxicity Detection Bias (2025.naacl-long)
Copied to clipboard
Samuel Bell, Mariano Coria Meglioli, Megan Richards, Eduardo Sánchez, Christophe Ropers, Skyler Wang, Adina Williams, Levent Sagun, Marta R. Costa-jussà
| Challenge: | Text toxicity detection systems produce disproportionate rates of false positives on demographic groups . toxicity classification systems often misinterpret benign group mentions as toxic . |
| Approach: | They use group annotations to compare text-based and speech-based toxicity detection systems. |
| Outcome: | The results show that access to speech data supports reduced bias against group mentions . the authors recommend improving classifiers, rather than transcription pipelines if possible . |
2M-BELEBELE: Highly Multilingual Speech and American Sign Language Comprehension Dataset Download PDF (2025.findings-acl)
Copied to clipboard
Marta R. Costa-jussà, Bokai Yu, Pierre Andrews, Belen Alastruey, Necati Cihan Camgoz, Joe Chuang, Jean Maillard, Christophe Ropers, Arina Turkatenko, Carleigh Wood
| Challenge: | We extend the BELEBELE dataset to speech and sign, and extend the Automatic Speech Recognition Benchmark, FLEURS, by 20%. |
| Approach: | They extend the BELEBELE and FLEURS speech comprehension datasets to speech and sign . they evaluate the datasets for 5-shot and zero-shot settings and find that the accuracy is 10% lower than reading comprehension. |
| Outcome: | The proposed dataset covers 91 spoken languages and one sign language (ASL) it also extends the Automatic Speech Recognition Benchmark, FLEURS, by 20% across languages. |
Enhancing Word Embeddings with Knowledge Extracted from Lexical Resources (2020.acl-srw)
Copied to clipboard
| Challenge: | In this paper, we present an effective method for semantic specialization of word vector representations. |
| Approach: | They propose a method for semantic specialization of word vector representations using BabelNet. |
| Outcome: | The proposed method improves on word similarity and dialog state tracking tasks. |